EDA of star dataset using Pandas Profiling

Dataset statistics

Number of variables	12
Number of observations	1100
Missing cells	0
Missing cells (%)	0.0%
Duplicate rows	0
Duplicate rows (%)	0.0%
Total size in memory	103.2 KiB
Average record size in memory	96.1 B

Variable types

NUM	10
CAT	2

Warnings

`Bolo MagMbol` is highly correlated with `Abs MagMv`	High correlation
`Abs MagMv` is highly correlated with `Bolo MagMbol`	High correlation
`RadiusRstar/Rsun` is highly correlated with `LuminosityLstar/Lsun`	High correlation
`LuminosityLstar/Lsun` is highly correlated with `RadiusRstar/Rsun`	High correlation
`df_index` has unique values	Unique
`StellarType` has unique values	Unique

Reproduction

Analysis started	2020-12-11 00:25:00.706294
Analysis finished	2020-12-11 00:25:23.229245
Duration	22.52 seconds
Software version	pandas-profiling v2.9.0
Download configuration	config.yaml

df_index
Real number (ℝ_≥0)

UNIQUE

Distinct	1100
Distinct (%)	100.0%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	618.7072727
Minimum	1
Maximum	1235
Zeros	0
Zeros (%)	0.0%
Memory size	8.6 KiB

Quantile statistics

Minimum	1
5-th percentile	61.95
Q1	309.75
median	618.5
Q3	928.25
95-th percentile	1175.05
Maximum	1235
Range	1234
Interquartile range (IQR)	618.5

Descriptive statistics

Standard deviation	357.3266268
Coefficient of variation (CV)	0.5775374601
Kurtosis	-1.200770317
Mean	618.7072727
Median Absolute Deviation (MAD)	309.5
Skewness	-0.0006035148592
Sum	680578
Variance	127682.3182
Monotocity	Strictly increasing

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
1235	1	0.1%
415	1	0.1%
408	1	0.1%
409	1	0.1%
410	1	0.1%
411	1	0.1%
412	1	0.1%
413	1	0.1%
416	1	0.1%
386	1	0.1%
Other values (1090)	1090	99.1%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
1	1	0.1%
2	1	0.1%
3	1	0.1%
4	1	0.1%
5	1	0.1%

Value	Count	Frequency (%)
1235	1	0.1%
1234	1	0.1%
1233	1	0.1%
1232	1	0.1%
1231	1	0.1%

Abs MagMv
Real number (ℝ)

HIGH CORRELATION

Distinct	190
Distinct (%)	17.3%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	-1.213636364
Minimum	-9.5
Maximum	19
Zeros	7
Zeros (%)	0.6%
Memory size	8.6 KiB

Quantile statistics

Minimum	-9.5
5-th percentile	-9.3
Q1	-6.5
median	-3.8
Q3	3.1
95-th percentile	14.2
Maximum	19
Range	28.5
Interquartile range (IQR)	9.6

Descriptive statistics

Standard deviation	7.149003981
Coefficient of variation (CV)	-5.890565078
Kurtosis	0.2212808114
Mean	-1.213636364
Median Absolute Deviation (MAD)	3.7
Skewness	1.054579862
Sum	-1335
Variance	51.10825792
Monotocity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
-9.3	36	3.3%
-5.5	34	3.1%
-7	28	2.5%
3.2	27	2.5%
-5.1	25	2.3%
-6.5	24	2.2%
-4.7	22	2.0%
-2.5	19	1.7%
3.3	18	1.6%
-5.4	17	1.5%
Other values (180)	850	77.3%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
-9.5	7	0.6%
-9.4	13	1.2%
-9.3	36	3.3%
-9.2	15	1.4%
-9.1	15	1.4%

Value	Count	Frequency (%)
19	8	0.7%
18.3	4	0.4%
17.9	4	0.4%
17.5	3	0.3%
16.8	8	0.7%

Bolo CorrBC(Temp)
Real number (ℝ)

Distinct	136
Distinct (%)	12.4%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	-2.465527273
Minimum	-8.38
Maximum	-0.08
Zeros	0
Zeros (%)	0.0%
Memory size	8.6 KiB

Quantile statistics

Minimum	-8.38
5-th percentile	-6.93
Q1	-3.86
median	-1.99
Q3	-0.62
95-th percentile	-0.1
Maximum	-0.08
Range	8.3
Interquartile range (IQR)	3.24

Descriptive statistics

Standard deviation	2.082540225
Coefficient of variation (CV)	-0.8446632281
Kurtosis	0.2721092652
Mean	-2.465527273
Median Absolute Deviation (MAD)	1.61
Skewness	-0.8700099815
Sum	-2712.08
Variance	4.336973789
Monotocity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
-1.65	33	3.0%
-4.01	30	2.7%
-0.09	27	2.5%
-3.15	24	2.2%
-4.58	24	2.2%
-0.08	21	1.9%
-8.38	20	1.8%
-0.1	19	1.7%
-4.8	15	1.4%
-4.16	15	1.4%
Other values (126)	872	79.3%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
-8.38	20	1.8%
-8.3	12	1.1%
-7.55	6	0.5%
-7.02	9	0.8%
-6.93	15	1.4%

Value	Count	Frequency (%)
-0.08	21	1.9%
-0.09	27	2.5%
-0.1	19	1.7%
-0.11	11	1.0%
-0.12	10	0.9%

Bolo MagMbol
Real number (ℝ)

HIGH CORRELATION

Distinct	582
Distinct (%)	52.9%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	-3.679163636
Minimum	-17.38
Maximum	15.18
Zeros	0
Zeros (%)	0.0%
Memory size	8.6 KiB

Quantile statistics

Minimum	-17.38
5-th percentile	-12.484
Q1	-9.655
median	-5.85
Q3	1.6175
95-th percentile	11.19
Maximum	15.18
Range	32.56
Interquartile range (IQR)	11.2725

Descriptive statistics

Standard deviation	7.531126422
Coefficient of variation (CV)	-2.046966965
Kurtosis	-0.4848662686
Mean	-3.679163636
Median Absolute Deviation (MAD)	4.575
Skewness	0.7422590012
Sum	-4047.08
Variance	56.71786518
Monotocity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
-11.08	12	1.1%
-9.91	9	0.8%
10.7	8	0.7%
-8.85	8	0.7%
-7.15	7	0.6%
-9.51	7	0.6%
6.19	7	0.6%
8.73	6	0.5%
12.65	6	0.5%
14.7	6	0.5%
Other values (572)	1024	93.1%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
-17.38	4	0.4%
-16.03	3	0.3%
-15.33	1	0.1%
-15.28	4	0.4%
-14.86	3	0.3%

Value	Count	Frequency (%)
15.18	6	0.5%
14.7	6	0.5%
14.11	6	0.5%
13.5	6	0.5%
12.94	2	0.2%

Color IndexB-V
Real number (ℝ)

Distinct	126
Distinct (%)	11.5%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	0.7154363636
Minimum	-0.37
Maximum	2.4
Zeros	8
Zeros (%)	0.7%
Memory size	8.6 KiB

Quantile statistics

Minimum	-0.37
5-th percentile	-0.35
Q1	-0.27
median	0.72
Q3	1.43
95-th percentile	2.25
Maximum	2.4
Range	2.77
Interquartile range (IQR)	1.7

Descriptive statistics

Standard deviation	0.8980828284
Coefficient of variation (CV)	1.255293796
Kurtosis	-1.355364519
Mean	0.7154363636
Median Absolute Deviation (MAD)	0.865
Skewness	0.2216615765
Sum	786.98
Variance	0.8065527666
Monotocity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
-0.35	78	7.1%
-0.34	39	3.5%
-0.32	39	3.5%
-0.33	33	3.0%
1.49	27	2.5%
1.41	25	2.3%
-0.3	24	2.2%
-0.31	24	2.2%
2.4	20	1.8%
-0.06	16	1.5%
Other values (116)	775	70.5%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
-0.37	6	0.5%
-0.35	78	7.1%
-0.34	39	3.5%
-0.33	33	3.0%
-0.32	39	3.5%

Value	Count	Frequency (%)
2.4	20	1.8%
2.39	12	1.1%
2.26	9	0.8%
2.25	15	1.4%
2.17	5	0.5%

LuminosityLstar/Lsun
Real number (ℝ_≥0)

HIGH CORRELATION

Distinct	639
Distinct (%)	58.1%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	4913954.586
Minimum	6.73e-05
Maximum	711000000
Zeros	0
Zeros (%)	0.0%
Memory size	8.6 KiB

Quantile statistics

Minimum	6.73e-05
5-th percentile	0.00266
Q1	17.9
median	17350
Q3	576750
95-th percentile	7838500
Maximum	711000000
Range	711000000
Interquartile range (IQR)	576732.1

Descriptive statistics

Standard deviation	44828400.87
Coefficient of variation (CV)	9.122673009
Kurtosis	223.4465663
Mean	4913954.586
Median Absolute Deviation (MAD)	17349.9994
Skewness	14.55034464
Sum	5405350045
Variance	2.009585525e+15
Monotocity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
2150000	9	0.8%
0.00415	8	0.7%
0.265	6	0.5%
0.000105	6	0.5%
0.000693	6	0.5%
0.00182	6	0.5%
6.73e-05	6	0.5%
0.000315	6	0.5%
6.91	6	0.5%
437000	6	0.5%
Other values (629)	1035	94.1%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
6.73e-05	6	0.5%
0.000105	6	0.5%
0.00018	6	0.5%
0.000315	6	0.5%
0.00053	2	0.2%

Value	Count	Frequency (%)
711000000	4	0.4%
205000000	3	0.3%
108000000	1	0.1%
103000000	4	0.4%
69600000	3	0.3%

MassMstar/Msun
Real number (ℝ_≥0)

Distinct	254
Distinct (%)	23.1%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	24.94218182
Minimum	0.1
Maximum	160
Zeros	0
Zeros (%)	0.0%
Memory size	8.6 KiB

Quantile statistics

Minimum	0.1
5-th percentile	0.2
Q1	2.175
median	7.6
Q3	15.8
95-th percentile	130
Maximum	160
Range	159.9
Interquartile range (IQR)	13.625

Descriptive statistics

Standard deviation	41.29636978
Coefficient of variation (CV)	1.655683937
Kurtosis	2.710068057
Mean	24.94218182
Median Absolute Deviation (MAD)	5.95
Skewness	2.007523769
Sum	27436.4
Variance	1705.390157
Monotocity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
0.1	35	3.2%
0.2	27	2.5%
0.8	21	1.9%
0.7	21	1.9%
0.5	20	1.8%
0.3	18	1.6%
0.6	18	1.6%
0.9	17	1.5%
0.4	14	1.3%
2.1	14	1.3%
Other values (244)	895	81.4%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
0.1	35	3.2%
0.2	27	2.5%
0.3	18	1.6%
0.4	14	1.3%
0.5	20	1.8%

Value	Count	Frequency (%)
160	3	0.3%
159.7	3	0.3%
159.4	3	0.3%
159	3	0.3%
158.7	3	0.3%

RadiusRstar/Rsun
Real number (ℝ_≥0)

HIGH CORRELATION

Distinct	594
Distinct (%)	54.0%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	2701.506317
Minimum	0.00696
Maximum	231000
Zeros	0
Zeros (%)	0.0%
Memory size	8.6 KiB

Quantile statistics

Minimum	0.00696
5-th percentile	0.00944
Q1	3.0375
median	20.5
Q3	235.5
95-th percentile	7567
Maximum	231000
Range	230999.993
Interquartile range (IQR)	232.4625

Descriptive statistics

Standard deviation	16541.59291
Coefficient of variation (CV)	6.123099844
Kurtosis	136.6412544
Mean	2701.506317
Median Absolute Deviation (MAD)	20.117
Skewness	10.89935991
Sum	2971656.948
Variance	273624295.8
Monotocity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
0.00864	12	1.1%
20.2	11	1.0%
0.553	9	0.8%
15	9	0.8%
14.9	9	0.8%
19.8	9	0.8%
20.5	7	0.6%
0.00696	6	0.5%
12.5	6	0.5%
14.2	6	0.5%
Other values (584)	1016	92.4%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
0.00696	6	0.5%
0.00864	12	1.1%
0.00887	6	0.5%
0.00889	6	0.5%
0.0089	6	0.5%

Value	Count	Frequency (%)
231000	4	0.4%
103000	3	0.3%
87900	4	0.4%
68200	1	0.1%
51000	3	0.3%

StellarType
Categorical

UNIQUE

Distinct	1100
Distinct (%)	100.0%
Missing	0
Missing (%)	0.0%
Memory size	8.6 KiB

M3Ia	1
C1Ib	1
WN3IV	1
DC5	1
C4III	1
Other values (1095)	1095

Frequencies
Length

Common Values
Overview

Value	Count	Frequency (%)
M3Ia	1	0.1%
C1Ib	1	0.1%
WN3IV	1	0.1%
DC5	1	0.1%
C4III	1	0.1%
WC9Ib	1	0.1%
WN3III	1	0.1%
K4II	1	0.1%
N6V	1	0.1%
WC8V	1	0.1%
Other values (1090)	1090	99.1%

Frequencies of value counts

Unique

Unique	1100 ?
Unique (%)	100.0%

Histogram of lengths of the category

Length

Max length	6
Median length	4
Mean length	4.209090909
Min length	3

TempK
Real number (ℝ_≥0)

Distinct	171
Distinct (%)	15.5%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	14385.17545
Minimum	1990
Maximum	100000
Zeros	0
Zeros (%)	0.0%
Memory size	8.6 KiB

Quantile statistics

Minimum	1990
5-th percentile	2180
Q1	3640
median	5670
Q3	24380
95-th percentile	47600
Maximum	100000
Range	98010
Interquartile range (IQR)	20740

Descriptive statistics

Standard deviation	16380.70046
Coefficient of variation (CV)	1.138720936
Kurtosis	3.104995851
Mean	14385.17545
Median Absolute Deviation (MAD)	2879
Skewness	1.660563497
Sum	15823693
Variance	268327347.6
Monotocity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
3700	25	2.3%
30200	24	2.2%
50000	24	2.2%
1990	20	1.8%
36800	15	1.4%
41200	15	1.4%
45600	15	1.4%
4669	15	1.4%
2180	15	1.4%
34600	15	1.4%
Other values (161)	917	83.4%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
1990	20	1.8%
2000	12	1.1%
2167	9	0.8%
2180	15	1.4%
2288	5	0.5%

Value	Count	Frequency (%)
100000	6	0.5%
50400	6	0.5%
50000	24	2.2%
47800	15	1.4%
47600	9	0.8%

color_decimal
Real number (ℝ_≥0)

Distinct	379
Distinct (%)	34.5%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	14392271.97
Minimum	9479935
Maximum	16776445
Zeros	0
Zeros (%)	0.0%
Memory size	8.6 KiB

Quantile statistics

Minimum	9479935
5-th percentile	9743103
Q1	10911231
median	16755508
Q3	16765601
95-th percentile	16773856.45
Maximum	16776445
Range	7296510
Interquartile range (IQR)	5854370

Descriptive statistics

Standard deviation	2899784.174
Coefficient of variation (CV)	0.2014820301
Kurtosis	-1.431727844
Mean	14392271.97
Median Absolute Deviation (MAD)	18623
Skewness	-0.5975208634
Sum	1.583149917e+10
Variance	8.408748258e+12
Monotocity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
9479935	24	2.2%
16751872	21	1.9%
9743103	21	1.9%
9611519	21	1.9%
9940223	15	1.4%
10203391	15	1.4%
10071807	15	1.4%
16764047	12	1.1%
10334975	12	1.1%
10334719	12	1.1%
Other values (369)	932	84.7%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
9479935	24	2.2%
9545727	3	0.3%
9611519	21	1.9%
9611775	3	0.3%
9742847	3	0.3%

Value	Count	Frequency (%)
16776445	1	0.1%
16776439	3	0.3%
16775931	1	0.1%
16775929	1	0.1%
16775420	2	0.2%

star_family
Categorical

Distinct	13
Distinct (%)	1.2%
Missing	0
Missing (%)	0.0%
Memory size	8.6 KiB

W	160
O	80
B	80
K	80
C	80
Other values (8)	620

Frequencies
Length

Common Values
Overview

Value	Count	Frequency (%)
W	160	14.5%
O	80	7.3%
B	80	7.3%
K	80	7.3%
C	80	7.3%
R	80	7.3%
G	80	7.3%
F	80	7.3%
S	80	7.3%
M	80	7.3%
Other values (3)	220	20.0%

Frequencies of value counts

Unique

Unique	0 ?
Unique (%)	0.0%

Histogram of lengths of the category

Length

Max length	1
Median length	1
Mean length	1
Min length	1

Pearson's r

The Pearson's correlation coefficient (r) is a measure of linear correlation between two variables. It's value lies between -1 and +1, -1 indicating total negative linear correlation, 0 indicating no linear correlation and 1 indicating total positive linear correlation. Furthermore, r is invariant under separate changes in location and scale of the two variables, implying that for a linear function the angle to the x-axis does not affect r.

To calculate r for two variables X and Y, one divides the covariance of X and Y by the product of their standard deviations.

Spearman's ρ

The Spearman's rank correlation coefficient (ρ) is a measure of monotonic correlation between two variables, and is therefore better in catching nonlinear monotonic correlations than Pearson's r. It's value lies between -1 and +1, -1 indicating total negative monotonic correlation, 0 indicating no monotonic correlation and 1 indicating total positive monotonic correlation.

To calculate ρ for two variables X and Y, one divides the covariance of the rank variables of X and Y by the product of their standard deviations.

Kendall's τ

Similarly to Spearman's rank correlation coefficient, the Kendall rank correlation coefficient (τ) measures ordinal association between two variables. It's value lies between -1 and +1, -1 indicating total negative correlation, 0 indicating no correlation and 1 indicating total positive correlation.

To calculate τ for two variables X and Y, one determines the number of concordant and discordant pairs of observations. τ is given by the number of concordant pairs minus the discordant pairs divided by the total number of pairs.

Phik (φk)

Phik (φk) is a new and practical correlation coefficient that works consistently between categorical, ordinal and interval variables, captures non-linear dependency and reverts to the Pearson correlation coefficient in case of a bivariate normal input distribution. There is extensive documentation available here.

Count
Matrix

First rows

	df_index	Abs MagMv	Bolo CorrBC(Temp)	Bolo MagMbol	Color IndexB-V	LuminosityLstar/Lsun	MassMstar/Msun	RadiusRstar/Rsun	StellarType	TempK	color_decimal	star_family
0	1	-9.5	-4.58	-14.08	-0.35	34100000.0	160.0	80.2	O0Ia0	50000.0	9479935	O
1	2	-6.7	-4.58	-11.28	-0.35	2590000.0	150.0	22.1	O0Ia	50000.0	9479935	O
2	3	-6.5	-4.58	-11.08	-0.35	2150000.0	140.0	20.2	O0Ib	50000.0	9479935	O
3	4	-6.5	-4.58	-11.08	-0.35	2150000.0	130.0	20.2	O0II	50000.0	9479935	O
4	5	-6.5	-4.58	-11.08	-0.35	2150000.0	120.0	20.2	O0III	50000.0	9479935	O
5	6	-6.0	-4.58	-10.58	-0.35	1360000.0	110.0	16.0	O0IV	50000.0	9479935	O
6	7	-5.9	-4.58	-10.48	-0.35	1240000.0	100.0	15.3	O0V	50000.0	9479935	O
7	8	-5.6	-4.58	-10.18	-0.35	940000.0	60.0	13.3	O0VI	50000.0	9479935	O
8	10	-9.4	-4.43	-13.83	-0.35	27100000.0	159.7	78.8	O1Ia0	47600.0	9611519	O
9	11	-6.7	-4.43	-11.13	-0.35	2250000.0	149.3	22.7	O1Ia	47600.0	9611519	O

Last rows

	df_index	Abs MagMv	Bolo CorrBC(Temp)	Bolo MagMbol	Color IndexB-V	LuminosityLstar/Lsun	MassMstar/Msun	RadiusRstar/Rsun	StellarType	TempK	color_decimal	star_family
1090	1226	10.2	-7.55	2.65	-0.37	6.910000	1.1	0.00902	DZ0	100000.0	10203903	D
1091	1227	10.8	-4.61	6.19	-0.35	0.265000	0.9	0.00696	DZ1	50400.0	10466815	D
1092	1228	11.4	-2.67	8.73	-0.27	0.025500	0.8	0.00864	DZ2	25200.0	11058687	D
1093	1229	11.9	-1.60	10.30	-0.20	0.006020	0.7	0.00944	DZ3	16800.0	11781631	D
1094	1230	12.5	-0.90	11.60	-0.13	0.001820	0.6	0.00922	DZ4	12600.0	12636159	D
1095	1231	13.1	-0.45	12.65	-0.07	0.000693	0.5	0.00890	DZ5	10080.0	13622015	D
1096	1232	13.7	-0.20	13.50	0.09	0.000315	0.4	0.00864	DZ6	8400.0	14739199	D
1097	1233	14.2	-0.09	14.11	0.34	0.000180	0.3	0.00889	DZ7	7200.0	15987711	D
1098	1234	14.8	-0.10	14.70	0.55	0.000105	0.2	0.00887	DZ8	6300.0	16775157	D
1099	1235	15.4	-0.22	15.18	0.74	0.000067	0.1	0.00898	DZ9	5600.0	16773089	D

Overview

Variables

Interactions

Correlations

Pearson's r

Spearman's ρ

Kendall's τ

Phik (φk)

Missing values

Sample

First rows

Last rows